6 research outputs found
Learning Gait Representation from Massive Unlabelled Walking Videos: A Benchmark
Gait depicts individuals' unique and distinguishing walking patterns and has
become one of the most promising biometric features for human identification.
As a fine-grained recognition task, gait recognition is easily affected by many
factors and usually requires a large amount of completely annotated data that
is costly and insatiable. This paper proposes a large-scale self-supervised
benchmark for gait recognition with contrastive learning, aiming to learn the
general gait representation from massive unlabelled walking videos for
practical applications via offering informative walking priors and diverse
real-world variations. Specifically, we collect a large-scale unlabelled gait
dataset GaitLU-1M consisting of 1.02M walking sequences and propose a
conceptually simple yet empirically powerful baseline model GaitSSB.
Experimentally, we evaluate the pre-trained model on four widely-used gait
benchmarks, CASIA-B, OU-MVLP, GREW and Gait3D with or without transfer
learning. The unsupervised results are comparable to or even better than the
early model-based and GEI-based methods. After transfer learning, our method
outperforms existing methods by a large margin in most cases. Theoretically, we
discuss the critical issues for gait-specific contrastive framework and present
some insights for further study. As far as we know, GaitLU-1M is the first
large-scale unlabelled gait dataset, and GaitSSB is the first method that
achieves remarkable unsupervised results on the aforementioned benchmarks. The
source code of GaitSSB will be integrated into OpenGait which is available at
https://github.com/ShiqiYu/OpenGait
GPGait: Generalized Pose-based Gait Recognition
Recent works on pose-based gait recognition have demonstrated the potential
of using such simple information to achieve results comparable to
silhouette-based methods. However, the generalization ability of pose-based
methods on different datasets is undesirably inferior to that of
silhouette-based ones, which has received little attention but hinders the
application of these methods in real-world scenarios. To improve the
generalization ability of pose-based methods across datasets, we propose a
\textbf{G}eneralized \textbf{P}ose-based \textbf{Gait} recognition
(\textbf{GPGait}) framework. First, a Human-Oriented Transformation (HOT) and a
series of Human-Oriented Descriptors (HOD) are proposed to obtain a unified
pose representation with discriminative multi-features. Then, given the slight
variations in the unified representation after HOT and HOD, it becomes crucial
for the network to extract local-global relationships between the keypoints. To
this end, a Part-Aware Graph Convolutional Network (PAGCN) is proposed to
enable efficient graph partition and local-global spatial feature extraction.
Experiments on four public gait recognition datasets, CASIA-B, OUMVLP-Pose,
Gait3D and GREW, show that our model demonstrates better and more stable
cross-domain capabilities compared to existing skeleton-based methods,
achieving comparable recognition results to silhouette-based ones. Code is
available at https://github.com/BNU-IVC/FastPoseGait.Comment: ICCV Camera Read
FastPoseGait: A Toolbox and Benchmark for Efficient Pose-based Gait Recognition
We present FastPoseGait, an open-source toolbox for pose-based gait
recognition based on PyTorch. Our toolbox supports a set of cutting-edge
pose-based gait recognition algorithms and a variety of related benchmarks.
Unlike other pose-based projects that focus on a single algorithm, FastPoseGait
integrates several state-of-the-art (SOTA) algorithms under a unified
framework, incorporating both the latest advancements and best practices to
ease the comparison of effectiveness and efficiency. In addition, to promote
future research on pose-based gait recognition, we provide numerous pre-trained
models and detailed benchmark results, which offer valuable insights and serve
as a reference for further investigations. By leveraging the highly modular
structure and diverse methods offered by FastPoseGait, researchers can quickly
delve into pose-based gait recognition and promote development in the field. In
this paper, we outline various features of this toolbox, aiming that our
toolbox and benchmarks can further foster collaboration, facilitate
reproducibility, and encourage the development of innovative algorithms for
pose-based gait recognition. FastPoseGait is available at
https://github.com//BNU-IVC/FastPoseGait and is actively maintained. We will
continue updating this report as we add new features.Comment: 10 pages, 4 figure
Dense Feature Aggregation and Pruning for RGBT Tracking
How to perform effective information fusion of different modalities is a core
factor in boosting the performance of RGBT tracking. This paper presents a
novel deep fusion algorithm based on the representations from an end-to-end
trained convolutional neural network. To deploy the complementarity of features
of all layers, we propose a recursive strategy to densely aggregate these
features that yield robust representations of target objects in each modality.
In different modalities, we propose to prune the densely aggregated features of
all modalities in a collaborative way. In a specific, we employ the operations
of global average pooling and weighted random selection to perform channel
scoring and selection, which could remove redundant and noisy features to
achieve more robust feature representation. Experimental results on two RGBT
tracking benchmark datasets suggest that our tracker achieves clear
state-of-the-art against other RGB and RGBT tracking methods.Comment: arXiv admin note: text overlap with arXiv:1811.0985
Weighted Channel Dropout for Regularization of Deep Convolutional Neural Network
In this work, we propose a novel method named Weighted Channel Dropout (WCD) for the regularization of deep Convolutional Neural Network (CNN). Different from Dropout which randomly selects the neurons to set to zero in the fully-connected layers, WCD operates on the channels in the stack of convolutional layers. Specifically, WCD consists of two steps, i.e., Rating Channels and Selecting Channels, and three modules, i.e., Global Average Pooling, Weighted Random Selection and Random Number Generator. It filters the channels according to their activation status and can be plugged into any two consecutive layers, which unifies the original Dropout and Channel-Wise Dropout. WCD is totally parameter-free and deployed only in training phase with very slight computation cost. The network in test phase remains unchanged and thus the inference cost is not added at all. Besides, when combining with the existing networks, it requires no re-pretraining on ImageNet and thus is well-suited for the application on small datasets. Finally, WCD with VGGNet-16, ResNet-101, Inception-V3 are experimentally evaluated on multiple datasets. The extensive results demonstrate that WCD can bring consistent improvements over the baselines
GaitEdge: Beyond Plain End-to-end Gait Recognition for Better Practicality
Gait is one of the most promising biometrics to identify individuals at a
long distance. Although most previous methods have focused on recognizing the
silhouettes, several end-to-end methods that extract gait features directly
from RGB images perform better. However, we demonstrate that these end-to-end
methods may inevitably suffer from the gait-irrelevant noises, i.e., low-level
texture and colorful information. Experimentally, we design the cross-domain
evaluation to support this view. In this work, we propose a novel end-to-end
framework named GaitEdge which can effectively block gait-irrelevant
information and release end-to-end training potential. Specifically, GaitEdge
synthesizes the output of the pedestrian segmentation network and then feeds it
to the subsequent recognition network, where the synthetic silhouettes consist
of trainable edges of bodies and fixed interiors to limit the information that
the recognition network receives. Besides, GaitAlign for aligning silhouettes
is embedded into the GaitEdge without losing differentiability. Experimental
results on CASIA-B and our newly built TTG-200 indicate that GaitEdge
significantly outperforms the previous methods and provides a more practical
end-to-end paradigm. All the source code are available at
https://github.com/ShiqiYu/OpenGait.Comment: 16 pages, 7 figures, accepted by ECCV202